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Outline 

• K-SVD and dictionary learning 

• Linear Blend Skinning 

• Brief survey on automatic skinning and compression 

• Dictionary learning for LBS 

• Two-layer sparse compression of Le & Deng. 

• This talk is about compressing skinned animations. 
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Representing Signals 


• We represent signals as linear combinations of things we already know - 
the 'basis' 





X (Xq + 
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Orthonormal Bases (ONBs) 

• The simplest way to represent signals is using a set of orthonormal bases 
+ 00 

0 i 3= j 

1 i=j 


bi(t)bj(t ) dt = 


— 00 
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Example ONBs 

• Fourier Basis 

b k (t) = e i2pkt 



• Wavelets 

b min (t) = a~ m/2 x(a~ m t - 

I 
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• Gabor Functions 

b k , n (t ) = a)(t - bn)e l2pkt 





• Contourlet 

bj,k, n(0 = hj.kit — 1 S fc n) 
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Benefits of ONB 

• Analytic formulations 

• Well understood mathematical properties 


Fast and simple algorithms for projection 
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Problems with ONB 

• One-size-fits all - not data adaptive 

• Global support cannot adapt to data locally 

• Fourier support is infinite, SH support spans the sphere 

• Try using Fourier to represent a step-function 

• Not sparse - very few zero coefficients 

• Not additive - relies on destructive cancellation. 
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Gibb's Ringing 


Si(x} 
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Fourier and SH 



20 


25 
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What is Overcomplete Dictionary? 

• Overcomplete means the dictionary has more atoms 
(columns) than the minimum required for the 
dimension of the signal 

• In 3D, an ONB only needs 3 basis 

• A 3D dictionary can have dozens or hundreds 
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The Sparse Signal Model 

K 

◄ ► 

N ■■■■■■■■■■■■■■■■ 

' ' l ■■■■■■■■■■■■■■■■ « 

A fixed dictionary 
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resulting 

signal 


a 


Sparse 
vector of 
coefficients 
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Why so many atoms? 

• More atoms give our algorithm a better chance to 
find a small subset that matches a given signal 

• Let's look at some patches from Barbara 
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Patches from Barbara 
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Domain Specific Compression 


• Just 550 bytes per image 

1. Original 

2 . JPEG 

3. JPEG2000 

4. PCA 

5 . KSVD per block 
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Project onto Dictionaries 

• Overcomplete and non-orthogonal 

• interactions among atoms cannot be ignored 

• How do we project? 

• Sparse Coding problem 
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Greedy Methods 

Matching Pursuit 

1. Set the residual r = x 

2 . Find an unselected atom 
that best matches the 
residual ||Da - r|| 

3. Re-calculate the residual 
from matched atoms 

r — x — Da 

4. Repeat until ||r|| < e 
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Orthogonal Matching Pursuit (OMP) 

• Add an Orthogonal Projection to the residual calculation 

1. set / := {0}, r := X, Y := 0 

2 . while (stopping test false) do 

3. k := argmax|<i£r| 

k 

4. I := (/, k) 

5. Yl ~ (D,) + X 

6. r := x — D iYi 

end while 


7 . 
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What is Dictionary Learning? 

• select a few atoms for each signal - e.g. OMP 

• Adjust the atoms to better fit those signals 

• Repeat 
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K-SVD 

• Is one of the well known dictionary learning 
methods 

• Check out our GDC2013 talk 

• our GDC13 slides "OMP and K-SVD for Sparse Coding" 

• See Jim's talk just before this session 

• Miral's Online Learning is the other. 
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Overcomplete Dictionary Recap 

• Importance of overcomplete dictionaries 

• OMP for efficient projection onto dictionaries 

• K-SVD for learning a better dictionary using samples 
from the real data 
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Part 2: 


• blank 
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Skinning 
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Linear Blend Skinning 

^ i = T} 1 B l 1 Wi j (RjPj + Tj) 

• Pi is the position for the ith vertex of the rest pose 

• Wij > 0 and sums to one (affinity). The non-negative 
constraint makes the blend additive. The affinity 
constraint prevents over-fitting and artifacts. 

• R; usually is orthogonal to avoid shearing or scaling 

• |B| is the number of weights (usually <= 6) 
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Blend Skinning on GPU 


JUUUUULJlJLJLJLJLJlJLJ^ 
IJUUUULJU 
JUUUUUL 
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LBS on GPUs 

• w ij typically very sparse - 4-6 weights or less per- 
vertex 

• Ideally a group of vertices all have the same weights 
to avoid thread divergence or splitting drawcalls 

• These are fairly serious constraints 

a) Some vertices might need more weights - e.g. very 
smooth meshes or complex topology (hand) 




Poisson-based Weight Reduction of Animated Meshes [Landreneau and Schaefer 2010] 



Not reduced k Largest Geometric Smooth Weights Poisson 


♦> Discrete optimization: \{wij\wij / 0 }| < \K\,Mi 

— Impossible to find optimum solution 

- Very high cost for non-optimum solution 

• Fracture 

• Significant increase of computing cost: nK non-zero -> n(K+ 1) non-zero 
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K- Largest 


\ 


7\ 


Original 
(not reduced) 
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fracturing 



k Largest 
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K-Largest - normals 
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Vertex Normal in Shader 

rain^-^ | oti(Mci — q )| 2 

M 

Solving for the inverse transpose 

M ~ T = 4 ad i(X) a i c icr)(^ a i c i c f) 

i i 
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Magic 4 

• why 4 weights is too few to generate smooth 
weights 

• 4 vertices specifies an affine transform exactly. 

• simplices in 3D contains 4 vertices for barycentric 
coordinates. 
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Two-layer Sparse Compression of 
Dense-Weigkt Bllend Skinning 
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Two-Layer Sparse Compression, Le & Deng 2013 

• Use dictionary learning to compute a two-level 
compression using bones 

• Work with the weights of the bind-pose directly 
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Why Dictionary for LBS? 

• Why dictionary learning? 

• limitations of Orthonormal-basis e.g. eigen/PCA 

• Not adaptive 

• Not purely additive - i.e. negative weights (relies on cancellation) 

• No intuitive meaning - bones extracted cannot be used to tweak the 
model 



Dense-Weigkt Compression 


Input: Dense matrix 



to 

0 ) 

c 

o 

GO 



Vertices 
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Original Blend Skinning Model 



Master Bone Trans. 

Master Bone Blending 

Virtual Bone Trans. 

Virtual Bone Blending 

'4 Vertex Transformations 

Our Compressed Skinning Model 






Sparse Matrix Factorization — dictionary learning 





min A w 2 = min II DA - Will 
D,A d,a kn * 

Subject to: card(di) < c,\/i 

card(o;i) < 2, Vi 
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Sparse Matrix Factorization 




min Aw 2 

D,A 


. 1 
mm - — 

d,a kn 


DA-W \\l 


Subject to: card (di) < c, Vi c=max{card(w,)}+l 
card(o;i) < 2, Vi 


UNIVERSITYof HOUSTON 


Sparse Matrix Factorization 



min Aw 2 

D,A 


. 1 
mm - — 

d,a kn 


DA-W \\l 


Subject to: card(di) < c, Vi 
card(oii) < 2, Vi 


t-max { card(w;) } + 1 

{ n is very large 
card(A)=2n— ^min 
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Algorithm — Btocle coordinate descent 



Alternative update D and A 
(Block coordinate descent) 



Update D 




UNIVERSITYof HOUSTON 






Update Coefficients A 


Linear least square with 2 unknowns 

2 

min || d r (a z ) r + d s (ai) s - Wi || 2 s.t. (a t ) r + (ai) s = 1 

(“i)r 

Oi)s 



Use mesh smoothness 
assumption to quickly 
find the non-zero 
candidates (virtual bones) 
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REDUCTION WITHOUT EXAMPLE POSE 



lLANDRENEAU AND SCHAEFER 2010] (WITHOUT USING EXAMPLE POSE) 
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Analysis of Two-Layer Scheme 

• Use 100's of virtual bones means we are not limited to a 
sparse approximation to the original animation. 

• virtual bones act as a 'common subexpression' 

• e.g. think compute shader that writes to LDS. 

• Still enforce sparsity on VBs to control runtime cost and 
LDS usage - but k can be 100's. 

• Per-vertex weights are 

• very sparse (2 per vertex) and the same for all vertices 

• good for GPU. 
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Learning Virtual Bones 

• Virtual bones are learned from the dense vertex weights 
by block-coordinate-descent (BCD): 

Sparse coding search for a few good atoms among the 
input columns. Use that to project all the rest of the inputs. 

• given the sparse weights from above we 

seek to adjust the atoms to make them fit the inputs that 
needs them better - a series of small LS problems. 

• Similar to EM/Lloyd-Max 
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Sparse Coding 

Sparse coding: 

• insert the vertex with the largest L2 norm 

• add a few more vertex which has the smallest dot- 
product with the 1 st atom 

• solve the basis-pursuit with OMP (see K-SVD) or LARS. 

• solve 2x2 least-square prob. for to blend masters 
bones 
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Weight Map - matrix A 

• Weights and indices for each vertex to blend virtual 
bones 

• solving a small 2x2 linear system to minimize MSE: 

• argmin x \\Dx — 1 1 A 2 

• runtime per-vertex cost is just 2 dotp 

• no bone hierarchy to worry about 

• no warp divergence even for high valence vertices 
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Atom Updates 

Atom update: 

foreach vertex 

• update each atom to minimize error for the set of vertices that 
reference it (this is like K-SVD) 

• Miral's Online Dictionary Learning [Miral09] 
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Atom Updates 


• Precompute A and B 

• A = a t a T 

• B = ’Z t i = 1 x l a T 

• For all atoms 

'T~iPi ~ Da ]) + d j -eq(5) 


• U\ 


• dj 


maxi || Uj 


II 2,1) 


— eq(6) 


• Uj is thresholded to make sure # of non — 
zero is below the # of master bones 
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• youtube 


Live Demo 



Compression witk Example Poses 



Without using example pose 

— Minimize weights difference 


. 1 
min - — 

d,a kn 


DA - W || 


With using example poses 

— Minimize reconstruction error 


min — — Ei 2 

d,a 3 fn ^ 
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CM 


Usin Exemplar poses 
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Virtual Bones Distribution 

L»1 
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Recap 

• The two-level scheme can work with dense (hand painted) 
weights or example poses (blend shape?) 

• Only the vertex positions are needed 

• a fixed memory footprint and uniform per-vertex cost - GPU 

friendly 

• Combines the quality of dense skinning and the efficiencies of 
sparse-LBS. Animators can use blend-shapes or FFD more. 
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Recap 2 

• Besides it uses dictionary learning and modern 
sparsity methods - how cool is that? © 

• Last year we show how good dictionary learning is 
for compressing 2d images and 3d volumes 

• Now we see what it can do for animation. 

• Thank you! 
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Recap 3 

• Non-negative LS and Active-set Method (ASM) 

• Block-coordinate descent 

• Sparsity constraints 

• Ll relaxation and LO-norm constraints 

• Direct solving 

• These are all very useful tools. 
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Activision R&D is Hiring 

• Our group is hiring © 
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Appendix 

• Kabsch/Procrutes method - use SVD to compute the 
MSE minimum rotation of one point-set to another. 

• Kabsch algorithm 



